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(54) Image data encoding system and image inputting apparatus 

(57) An image data encoding system is disclosed, 
that comprises a discrete cosine transforming means 

(101) for discrete-cosine transforming the original 
image, an electronic watermark data embedding means 

(102) for embedding the electronic watermark data 

(103) in the data that has been transformed by the dis- 
crete cosine transforming means (101), and a data 
selecting means (106; 110) for selecting the output sig- 
nal (107) of the discrete cosine transforming means 
(101) or the output signal (108) of the electronic water- 
mark data embedding means (102). 

Another image data encoding system is also dis- 
closed, that comprises an electronic watermark embed- 
ding means (404) for embedding electronic watermark 

data selected from a plurality of types (408(0) 408(n)) 

of electronic watermark data to the digital image data 
(401), wherein at least one of the plurality of types 
(408(0), ...,408(n)) of electronic watermark data is pre- 
determined electronic watermark data that does not 
affect the digital image data (401) even if the electronic 
watermark data is embedded in the digital image data 
(401). 
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Description 

Background of the Invention 
Field of the Invention: 

The present invention relates to a digital image 
processing system and. in particular, to an image data 
encoding system for embedding identification data with 
special information (hereinafter, referred to as electronic 
watermark data) to a digital image. In addition, the 
present invention relates to an image inputting appara- 
tus for use in, for example, a personal computer and, in 
particular, to an image inputting apparatus equipped 
with an illegal copy prohibiting function. 

Description of the Related Art: 

In recent years, the act of illegally copying digital 
images causes a social problem. To prevent digital 
images from being illegally copied, an 

encryption system has been proposed. In this sys- 
tem, digital image data is encrypted. Only a reproducing 
system with a valid decryption key can reproduce the 
encrypted digital image data. However, in such a sys- 
tem, once encrypted data is decrypted, there is no way 
to prevent the data from being copied any more. 

The purpose of a conventional illegal copy prohibit- 
ing method for an image inputting apparatus was to pre- 
vent the instance of copying image data. 

Fig. 9 is a block diagram showing an example of an 
image inputting apparatus equipped with a conventional 
illegal copy prohibiting function. An input image is sup- 
plied to image pickup means 901, analog-to-digital con- 
verting means 902, converting means 903, quantizing 
means 904, and variable-length encoding means 905. 
After the input image is converted into compressed 
image data such as an MPEG data stream, the resultant 
data is supplied to scrambling means 906. Scrambling 
means 906 scrambles the input data and outputs com- 
pressed and scrambled image data. The compressed 
and scrambled image data can be reproduced only by 
an apparatus with a de-scrambling function. 

As explained above, in the conventional system, 
images are scrambled to be prevented from being ille- 
gally copied. 

In the conventional system, once scrambled 
images were descrambled, it was impossible to prevent 
them from being illegally copied. 

In addition to such a conventional system, in order 
to prevent bills and securities from being illegally cop- 
ied, a method for embedding identification information 
in pixel components of an image has been proposed in, 
for example, Japanese Patent Laid-Open Publication 
Nos. 4-351 164, 6-22062, and 6-221 19. 

In the method for embedding identification informa- 
tion to pixel components of an image, there was the dis- 
advantage that the identification information could be 



2 

easily forged and removed. 

Therefore, a method for embedding electronic 
watermark data in a digital image has been proposed to 
prevent digital images from being illegally used and cop- 
5 ied. 

There are two types of electronic watermark data 
for digital images, i.e. visible electronic watermark data 
and invisible electronic watermark data. 

The visible electronic watermark data is composed 

10 of special characters or symbols so that it can be recog- 
nized by visual sensation. Although the visible elec- 
tronic watermark data causes deterioration of the image 
quality, the user of the digital image can distinguish it 
from a forged one, whereby illegal circulation of bills or 

75 securities can be prevented. 

An example of a method for embedding visible elec- 
tronic watermark data in an electronic image is dis- 
closed in Japanese Patent Laid-Open Publication No. 8- 
241403. In this method, when visible electronic water- 

20 mark data is combined with an original image, only the 
brightness of pixels corresponding to an opaque portion 
of the electronic watermark data is varied, not color 
components. In this method, scaling values which vary 
the brightness components of the pixels are determined 

25 corresponding to color components, random numbers, 
pixel values of electronic watermark data, or the like. 

On the other hand, the invisible electronic water- 
mark data is embedded in an image in such a manner 
that the electronic watermark data does not affect the 

30 image quality. Thus, since the invisible electronic water- 
mark faintly deteriorates the image quality, the deterio- 
ration is not perceivable by visual sensation. When 
special information that identifies a copyright holder of a 
original image is embedded in the form of the electronic 

35 watermark data, even after the image has been illegally 
copied, the copyright holder of the image can be identi- 
fied by detecting the electronic watermark data. In addi- 
tion, in the case that information inhibiting duplication is 
embedded in a image in the form of electric watermark 

40 data, when a relevant reproducing unit such as VTR 
detects the information, the unit can inform the user that 
the duplication of the image is inhibited or the unit can 
prevent duplication of the image by activating duplica- 
tion inhibiting mechanism. 

45 As one method for embedding invisible electronic 
watermark data in a digital image, special information 
representing invisible electronic watermark is embed- 
ded in a portion where the information faintly affects the 
picture quality such as the least significant bits (LSBs) 

so of pixel data. However, in this method, it is easy to erase 
the electronic watermark data from the image. For 
example, with a low-pass filter, the information of LSBs 
of the pixel data can be removed. Additionally, in the 
image compressing process, redundant data that faintly 

55 affects the image quality is removed so as to reduce the 
data amount and the electric watermark data is embed- 
ded in the place where redundant data exists. Thus, 
when the image compressing process is performed, the 
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electronic watermark data is lost. Consequently, it is dif- 
ficult to detect the electronic watermark data of an 
image that has been compressed. 

To solve this problem, a method for transforming an 
image into frequency components and embedding elec- 
tronic watermark data in the frequency spectrum has 
been proposed (Nikkei Electronics, p. 13, No. 660, April 
22, 1996). In this method, since electronic watermark 
data is embedded in frequency components, even if an 
image process such as a compressing process or a fil- 
tering process is performed for an image, the electronic 
watermark data is not lost. In addition, when random 
numbers that follow a normal distribution are used as 
electronic watermark data, different pieces of electronic 
watermark data do not interfere with each other. Thus, it 
is difficult to destroy the electronic watermark data with- 
out largely deteriorating the image. 

Referring to Fig. 10, the method for embedding 
electronic watermark data in an image is performed as 
follows. First of all, a discrete cosine transforming 
means 1020 transforms an original image into fre- 
quency components. In the frequency components, n 

components are selected as f(1), f(2) f(n) according 

to amplitude order. Electronic watermark data pieces 
w(1), w(2), .... w(n) are extracted from random data fol- 
lowing a normal distribution with means = 0 and vari- 
ance = 1. An electronic watermark data embedding 
means 1030 calculates the following equation for each i: 

F(i) =f(i)+a |f(i)| • w(i), 

where 1 ^ I s n and where a is a scaling factor. Finally, 
image data in which electronic watermark data has 
been embedded is obtained by transforming F(l) by 
inverse discrete cosine transform. 

The electronic watermark data is detected in the fol- 
lowing manner. In this case, it is assumed that the orig- 
inal image and electronic watermark data candidate set 
{w(i)} (where i = 1 , 2 n) are known. 

With reference to Fig. 11, a discrete cosine trans- 
forming means 1 120 transforms an image in which elec- 
tronic watermark data has been embedded into 

frequency components F(1), F(2) F(n). A discrete 

cosine transforming means 1110 transforms original 
image data into frequency components f(1 ), f(2), .... f(n). 
With f(i) and F(i), electronic watermark data estimated 
values W(i) are calculated and extracted by the follow- 
ing equation: 

W(i) = (F(i)-f(i))/f(i). 

Next, an inner product calculating means 1140 calcu- 
lates the statistical similarity of w(i) and W(i) by the fol- 
lowing equation: 

C = W * w / (WD * wD), 
where W = (W(1), W(2), .... W(n)); w = (w(1), w(2) 



w(n)); WD = absolute value of vector W; and wD = abso- 
lute value of vector w. A statistical similarity determining 
means 1160 determines that relevant electronic water- 
mark data has been embedded in a relevant image 

5 when the value of C is equal to or larger than a prede- 
termined value. 

If the copyright holder of images embeds electronic 
watermark data in the images, the electronic watermark 
data is effective to check out images that the holder 

10 doubts is illegally copied. Fig. 12 is a block diagram 
showing an image data encoding system with such an 
electronic watermark data embedding means according 
to a related art reference. Discrete cosine transforming 
means 1201 orthogonally transforms the original image 

is data in time domain into data in frequency domain. 
Electronic data embedding means 1202 embeds elec- 
tronic watermark data 1203 in the data in frequency 
domain. Quantizing means 1204 quantizes the data in 
which the electronic watermark data has been embed- 

20 ded. Encoding means 1 205 encodes the quantized data 
and outputs the resultant MPEG data. 

The aforementioned conventional encoding system 
always embeds electronic watermark data in a relevant 
image. Although the image faintly deteriorates as the 

25 electronic watermark data is embedded in frequency 
components, it is not that the image does not at all dete- 
riorate. Therefore, another image encoding system hav- 
ing no means for embedding electric watermark data is 
required when image should not be embedded with 

30 electric watermark data, especially when the quality of 
the image should be valued. 

Summary of the Invention 

35 An object of the present invention is to provide an 
encoding system that generates not only encoded data 
of image in which electric watermark data is embedded 
but also encoded data of image in which electric water- 
mark data is not embedded. 

40 Another object of the present invention is to provide 
an image inputting apparatus that generates not only 
encoded data of image in which electric watermark data 
is embedded but also encoded data of image in which 
electric watermark data is not embedded. 

45 According to one aspect of the present invention, 
there is provided an image data encoding system for 
embedding electronic watermark data to an original 
image, which comprises: discrete cosine transforming 
means for discrete-cosine transforming the original 

so image; electronic watermark data embedding means for 
embedding the electronic watermark data in the data 
that has been transformed by the discrete cosine trans- 
forming means; and data selecting means for selecting 
the output signal of the discrete cosine transforming 

55 means or the output signal of the electronic watermark 
data embedding means. 

The image data encoding system further com- 
prises: a flip-flop connected to the data selecting 
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means, wherein the data selecting means selects the 
output signal of the discrete cosine transforming means 
or the output signal of the electronic watermark data 
embedding means corresponding to information stored 
in the flip-flop. 

In the image data encoding system, the data select- 
ing means selects the output signal of the discrete 
cosine transforming means or the output signal of the 
electronic watermark data embedding means corre- 
sponding to an external signal. 

According to another aspect of the present inven- 
tion, there is provided an image data encoding system 
for encoding digital image data in a predetermined 
encoding manner and outputting the resultant image 
data, which comprises: an electronic watermark 
embedding means for embedding electronic watermark 
data selected from a plurality of types of electronic 
watermark data to the digital image data, wherein at 
least one of the plurality of types of electronic water- 
mark data is predetermined electronic watermark data 
that does not affect the digital image data even if 
embedded in the digital image data. 

In the image data encoding system, the predeter- 
mined electronic watermark data is composed of other 
than random numbers generated by an algorithm corre- 
sponding to a normal distribution. 

The image data encoding system further com- 
prises: transforming means for transforming the digital 
image data into frequency components and outputting 
the resultant data to the electronic watermark data 
embedding means; quantizing means for quantizing the 
data in which electronic watermark data has been 
embedded by the electronic watermark data embedding 
means; and a variable-length encoding means for 
encoding output data of the quantizing means into vari- 
able-length code- 
According to still another aspect of the present 
invention, there is provided an image data encoding 
system for encoding digital image data in a predeter- 
mined manner and outputting the resultant data, com- 
prising: a plurality of electronic watermark data tables 
having a plurality of types of electronic watermark data 
for identifying the digital image data; an electronic 
watermark data selecting means for selecting one of the 
electronic watermark data tables; and an electronic 
watermark data embedding means for embedding the 
selected type of electronic watermark data in the digital 
image data, wherein at least one of the electronic water- 
mark data tables has a predetermined electronic water- 
mark data that does not affect the digital image data 
even if embedded in the digital image data. 

In the image data encoding system, the predeter- 
mined electronic watermark data is composed of other 
than random numbers generated by an algorithm corre- 
sponding to a normal distribution. 

The image data encoding system further comprises 
transforming means for transforming the digital image 
data into frequency components and outputting the 



resultant data to the electronic watermark data embed- 
ding means; a quantizing means for quantizing the data 
in which electronic watermark data has been embedded 
by the electronic watermark data embedding means; 

s and a variable-length encoding means for encoding out- 
put data of the quantizing means into variable-length 
code. In the image data encoding system, the predeter- 
mined electronic watermark data is composed of other 
than random numbers generated by an algorithm corre- 

io sponding to a normal distribution. 

According to the further aspect of the present 
invention, there is provided an image inputting appara- 
tus, which comprises: image pickup means for obtaining 
an analog image signal; analog -to-digital converting 

15 means for converting the analog image signal obtained 
by the image pickup means into image data; transform- 
ing means for transforming the image data into data in 
first frequency domain; storing means for temporarily 
storing the data in the first frequency domain; identifica- 

20 tion data holding means for holding identification data; 
means for adding the identification data to the data in 
the first frequency domain and generating data in sec- 
ond frequency domain; and selecting means for select- 
ing either of the data in the first frequency domain and 

25 the data in the second frequency domain and outputting 
the selected data. 

In the image inputting apparatus, the transforming 
means is an orthogonal transforming means. 

The image inputting apparatus further comprises: 

30 compressing means for compressing and encoding the 
output signal of the selecting means. 

According to still further aspect of the present 
invention, there is provided an image inputting appara- 
tus, comprising: image pickup means for obtaining an 

35 analog image signal; analog-to<ligital converting 
means for converting the analog image signal obtained 
by the image pickup means into image data; transform- 
ing means for transforming the image data into data in 
first frequency domain; storing means for temporarily 

40 storing the image data; identification data holding 
means for holding identification data; means for adding 
the identification data to the data in the first frequency 
domain and generating data in second frequency 
domain; inverse-transforming means for inversely trans- 

45 forming the data in the second frequency domain into 
data in time domain; and selecting means for selecting 
either of the output signal of the inverse-transforming 
means and the output signal of the storing means. 

In the image inputting apparatus, the transforming 

so means is an orthogonal transforming means and the 
inverse-transforming means is an orthogonal inverse- 
transforming means. 

The image inputting apparatus, further comprises: 
compressing means for compressing and encoding the 

55 output signal of the selecting means. 

These and other objects, features and advantages 
of the present invention will become more apparent in 
light of the following detailed description of a best mode 
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embodiment thereof, as illustrated in the accompanying 
drawings. 

RriPf Description of Drawings 

Fig. 1 is a block diagram showing the structure of an 

image data encoding system according of a first 

embodiment of the present invention; 

Fig. 2 is a block diagram showing the structure of an 

image data encoding system according to a first 

example of the present invention; 

Fig. 3 is a block diagram showing the structure of an 

image data encoding system according to a second 

example of the present invention; 

Fig. 4 is a block diagram showing the structure of an 

image data encoding system according of a second 

embodiment of the present invention; 

Fig. 5 is a block diagram showing the structure of an 

image inputting apparatus according to a third 

embodiment of the present invention; 

Fig. 6 is a block diagram showing the structure of an 

image inputting apparatus according to a third 

example of the present invention; 

Fig. 7 is a schematic diagram for explaining an 

embedment of identification data in frequency 

domain according to the third example of the 

present invention; 

Fig. 8 is a block diagram showing the structure of an 
image inputting apparatus according to a fourth 
example of the present invention; 
Fig. 9 is a block diagram showing an example of the 
structure of a conventional image inputting appara- 
tus; 

Fig. 10 is a block diagram for explaining an elec- 
tronic watermark data embedding method accord- 
ing to a related art reference; 
Fig. 1 1 is a block diagram for explaining an elec- 
tronic watermark data detecting method according 
to a related art reference; and 
Fig. 12 is a block diagram showing the structure of 
an image data encoding system according to a 
related art reference. 

Description of Preferred Embodiments 

Next, with reference to the accompanying drawings, 
embodiments and examples of the present invention will 
be explained. 

First Embodiment of Present Invention 

With reference to Fig. 1, an image data encoding 
system according to a first embodiment of the present 
invention comprises discrete cosine transforming 
means 101. electronic watermark data embedding 
means 102, electronic watermark data 103, data select- 
ing means 106, quantizing means 104, and encoding 
means 105. Discrete cosine transforming means 101 



transforms input original image data in time domain into 
data in frequency domain. Electronic watermark data 
embedding means 102 embeds electronic watermark 
data 103 in the data in frequency domain. Data select- 

5 ing means 1 06 alternatively selects output signal 1 07 of 
discrete cosine transforming means 101 or output sig- 
nal 1 08 of electronic watermark data embedding means 
102. Quantizing means 104 quantizes data selected by 
data selecting means 106. Encoding means 105 

10 encodes the quantized data received from quantizing 
means 104 and generates MPEG data. 

Next, the operation of the system shown in Fig. 1 
will be explained. 

Discrete cosine transforming means 101 converts 

is the original image data in time domain into data in fre- 
quency domain. Electronic watermark data embedding 
means 102 embeds electronic watermark data 103 in 
the data in frequency domain. 

Output signal 108 of electronic watermark data 

20 embedding means 1 02 is supplied to one input terminal 
of data selecting means 106. Output signal 107 of dis- 
crete cosine transforming means 101 is supplied to an 
input terminal of electronic watermark data embedding 
means 102. In addition, output signal 107 is supplied to 

25 the other input terminal of data selecting means 106. 
When the electronic watermark data 103 should be 
embedded in the original image data, data selecting 
means 106 selects output signal 108. When the elec- 
tronic watermark data 103 should not be embedded in 

30 the original image data, data selecting means 106 
selects output signal 107. 

Quantizing means 104 quantizes the data selected 
by data selecting means 106. Encoding means 105 
encodes the quantized data and outputs MPEG data. 

35 

First Example of Present Invention 

Next, with reference to Fig. 2, a first example 
according to the first embodiment of the present inven- 
40 tion will be explained. 

With reference to Fig. 2, output signal 107 of dis- 
crete cosine transforming unit 101 or output signal 108 
of electronic watermark data embedding unit 102 is 
alternatively selected by selecting unit 110 that oper- 
as ates corresponding to information stored in flip-flop 111. 
When the electronic watermark data should not be 
embedded in the image data, a logic value "0" is stored 
in flip-flop 111. When the electronic watermark data 
should be embedded in the original image data, a logic 
so value "1 " is stored in flip-flop 111. 

Discrete cosine transforming unit 101 orthogonally 
transforms original image data in time domain into data 
in frequency domain. Electronic watermark data 
embedding unit 102 embeds electronic watermark data 
55 1 03 in the data in frequency domain. 

Output signal 108 of electronic watermark data 
embedding unit 102 is supplied to one input terminal of 
selecting unit 1 10. Output signal 107 of discrete cosine 
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transforming unit 101 is supplied to an input terminal of 
electronic watermark data embedding unit 102. In addi- 
tion, output signal 107 of discrete cosine transforming 
unit 101 is supplied to the other input terminal of select- 
ing unit 1 1 0. When the information of flip-flop 1 1 1 repre- 
sents the logical value '0', selecting unit 110 selects 
output signal 107. When the information of flip-flop 1 1 1 
represents the logical value T, selecting unit 110 
selects output signal 108. 

Quantizing unit 104 quantizes the data selected by 
selecting unit 110. Encoding unit 105 encodes the 
quantized data and outputs MPEG data. 

Second Example of Present Invention 

Next, with relerence to Fig. 3, a second example 
according to the first embodiment of the present inven- 
tion will be explained. 

Referring to Fig. 3, output signal 107 of discrete 
cosine transforming unit 101 and output signal 108 of 
electronic watermark data embedding unit 102 is alter- 
natively selected by selecting unit 110 corresponding to 
external signal 112. When electronic watermark data 
103 should not be embedded in image data, a logical 
value '0* is set to external signal 112. When electronic 
watermark data should be embedded in image data, a 
logical value T is designated to external signal 112. 

Discrete cosine transforming unit 101 orthogonally 
transforms original image data in time domain into data 
in frequency domain. Electronic watermark data 
embedding unit 102 embeds electronic watermark data 
103 in the data in frequency domain. 

Output signal 108 of electronic watermark data 
embedding unit 102 is supplied to one input terminal of 
selecting unit 110. Output signal 107 of discrete cosine 
transforming unit 101 is supplied to an input terminal of 
electronic watermark data embedding unit 102. In addi- 
tion, output signal 107 is supplied to the other input ter- 
minal of selecting unit 110. When external signal 112 
represents the logical value , 0\ selecting unit 110 
selects output signal 107. When external signal 112 
represents the logical value '1*. selecting unit 110 
selects output signal 108. 

Quantizing unit 104 quantizes the data selected by 
selecting unit 110. Encoding unit 105 encodes the 
quantized data and outputs MPEG data. 

Second Embodiment of Present Invention 

Next, with reference to Fig. 4, an image data encod- 
ing system according to a second embodiment of the 
present invention will be explained. 

Fig. 4 is a block diagram showing the structure of 
the image data encoding system according to the sec- 
ond embodiment of the present invention. In Fig. 4, the 
image data encoding system comprises discrete cosine 
transforming means 402, a plurality of electronic water- 
mark data tables 408(0), 408(1), 408(2)... 408(n), elec- 



tronic watermark data selecting unit 407, electronic 
watermark data embedding means 404, quantizing 
means 405, and encoding means 406. Discrete cosine 
transforming means 402 performs a discrete cosine 

5 transforming process for original image stream 401 to 
be encoded. Electronic watermark data tables 408(0), 

408(1), 408(2) 408(n) have respective electronic 

watermark data. Electronic watermark data selecting 
unit 407 selects one of electronic watermark data tables 

10 408(0), 408(1), 408(2), ... , 408(n), Electronic water- 
mark data embedding means 404 embeds electronic 
watermark data in the data that is received from discrete 
cosine transforming means 402 and then temporarily 
stored in buffer 410. Quantizing means 405 quantizes 

75 data received from electronic watermark data embed- 
ding means 404. Encoding means 406 encodes data 
received from quantizing means 405 into variable- 
length code and outputs resultant MPEG data 409. 
Among the plurality of electronic watermark data 

20 tables 408(0) to 408(n), watermark data table 408(0) 
has electronic data that does not affect digital image 
data. In other words, the electronic watermark data 
table 408(0) does not have random numbers generated 
by an algorithm of generating random numbers in a nor- 

25 mal distribution. On the other hand, electronic water- 
mark data tables 408(1) to 408(n) have random 
numbers generated by the algorithm. 

Next, the operation of the image data encoding sys- 
tem according to the second embodiment of the present 

30 invention will be explained. 

First of all, the case in which normal electronic 
watermark data is embedded in image data will be 
explained. Original image data 401 is extracted in the 
unit of (8 x 8 pixel) block. Discrete cosine transforming 

35 means 402 performs a discrete cosine transforming 
process for the extracted data and then transforms the 
data into frequency components. Electronic watermark 
data selecting means 407 selects electronic watermark 
data from one of the electronic watermark data tables 

40 408(1) to 408(n) except for electronic watermark data 
table 408(0) and outputs the selected electronic water- 
mark data to electronic watermark data embedding 
means 404. Electronic watermark data embedding 
means 404 embeds the selected electronic watermark 

45 data in the frequency components. Quantizing means 
405 quantizes data received from electronic watermark 
data embedding means 404. Encoding means 406 
encodes quantized data and outputs resultant MPEG 
data 409. 

so Next, the case in which encoded data correspond- 
ing to original data is required is explained. Similarly to 
the normal case, original image data 401 is extracted in 
the unit of (8 x 8 pixel) block corresponding to the con- 
ventional MPEG compressing process. Discrete cosine 

55 transforming means 402 performs a discrete cosine 
transforming process for the extracted data and then 
transforms the extracted data into frequency compo- 
nents. Electronic watermark data selecting means 407 
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selects electronic, watermark data that does not affect 
digital image data from the electronic watermark data 
table 408(0) and outputs the selected electronic water- 
mark data to electronic watermark data embedding 
means 404. Electronic watermark data embedding 
means 404 embeds the selected electronic watermark 
data in the frequency components. Quantizing means 

405 quantizes the data received from electronic water- 
mark data embedding means 404. Encoding means 

406 encodes the quantized data and outputs resultant 
MPEG data 409. 

Third Embodiment of Present Invention 

Next, with reference to Fig. 5, the basic structure of 
an image inputting apparatus according to a third 
embodiment of the present invention will be explained. 
With reference to Fig. 5, the image inputting apparatus 
comprises image pickup means 501, analog-to-digital 
converting means 502, transforming means 503, storing 
means 507, identification data holding means 510, iden- 
tification data embedding means 509, data selecting 
means 508, quantizing means 504, and encoding 
means 505. Image pickup means 501 picks up an exter- 
nal image. Analog-to-digital converting means 502 con- 
verts an analog signal of the picked-up image into digital 
image data. Transforming means 503 transforms the 
image data in space domain into data in frequency 
domain. Storing means 507 temporarily stores the 
image data in frequency domain. Identification data 
holding means 510 holds identification data. Identifica- 
tion data embedding means 509 embeds the identifica- 
tion data in the image data in frequency domain. Data 
selecting means 508 selects an output signal of storing 
means 507 or an output signal of identification data 
embedding means 509. Quantizing means 504 quan- 
tizes image data. Encoding means 505 encodes the 
quantized image data into variable-length code. 

Next, the operation of the image inputting appara- 
tus according to the third embodiment of the present 
invention will be explained. Image pickup means 501 
picks up an external image and outputs the analog sig- 
nal of the picked-up image. Analog-to-digital converting 
means 502 converts the analog signal into digital image 
data and outputs the digital image data. Transforming 
means 503 orthogonally transforms the image data in 
space domain into image data in frequency domain and 
outputs the resultant image data. Storing means 507 
temporarily stores the image data in frequency domain. 
Identification data holding means 51 0 holds and outputs 
identification data. Identification data embedding means 
509 embeds the identification data in the image data in 
frequency domain and outputs the resultant data. Data 
selecting means 508 selects an output signal of storing 
means 507 or an output signal of identification data 
embedding means 509 and outputs the selected signal. 
The quantizing means 504 quantizes the image data 
and outputs the resultant data. The encoding means 



505 encodes the quantized image data into variable- 
length code and outputs compressed image data. 

Third Example of P resent Invention 

5 

Next, with reference to Fig. 6, a third example 
according to the third embodiment of the present inven- 
tion will be explained. Referring to Fig. 6, CCD image 
pickup device 601 picks up an external image and out- 
10 puts the analog signal of the picked-up image. Analog- 
to-digital converting unit 602 converts the analog signal 
into digital image data and outputs the digital image 
data. 

Discrete cosine transforming unit 603 orthogonally 

15 transforms the image data in space domain into data in 
frequency domain. Buffer 607 temporarily stores image 
data in frequency domain. Identification data table 610 
holds and outputs identification data. Identification data 
embedding unit 609 embeds the identification data in 

20 the image data in frequency domain and outputs the 
resultant data. 

Selecting unit 608 alternatively selects an output 
signal of buffer 607 or an output signal of identification 
data embedding unit 609. When the output signal of 

25 buffer 607 is selected, the original image data is output. 
When the output signal of identification data embedding 
unit 609 is selected, image data in which the identifica- 
tion data is embedded is output. 

Quantizing unit 604 quantizes image data and out- 

30 puts the quantized image data. Variable-length encod- 
ing unit 605 encodes the quantized image data in 
variable-length code and outputs the resultant MPEG 
data. The MPEG data is supplied to for example a per- 
sonal computer, a storage medium processing unit 

35 (such as an optical magnetic medium), a network 
processing unit (that transmits the data to a network 
line), or a radio media processing unit (that transmits 
the data to a radio channel). 

Next, with reference to Fig. 7, an embedding 

40 method of identification data will be explained. When 
image data in space domain is orthogonally trans- 
formed into data in frequency domain by discrete cosine 
transforming unit 603, a frequency spectrum 701 shown 
in Fig. 7 is generated. Identification data table 610 out- 

45 puts the identification data with a frequency spectrum 
704 shown in Fig. 7. Frequency spectrum 704 is similar 
to the spectrum 701. When adding unit 702 adds fre- 
quency spectrum 704 of the identification data to fre- 
quency spectrum 701 of the original image, a frequency 

so spectrum 703 in which the identification data is embed- 
ded is obtained. 

To extract the identification data, a subtracting unit 
(not shown) extracts frequency spectrum 701 of the 
original image from frequency spectrum 703 in which 

55 the identification data is embedded and obtains fre- 
quency spectrum 704 of the identification data. Thus, 
the identification data can be easily extracted. 
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Fourth Example of Present Invention 

Next, with reference to Fig. 8, a fourth example 
according to the third embodiment of the present inven- 
tion will be explained. Referring to Fig. 8, CCD image 
pickup device 601 picks up an external image and out- 
puts the analog signal of the picked-up image. Analog- 
to-digital converting unit 602 converts the analog signal 
into digital image data and outputs the image data. 
Buffer 807 temporarily stores the digital image data. 

Discrete cosine transforming unit 603 orthogonally 
transforms image data in space domain into data in fre- 
quency domain and outputs the resultant data. Identifi- 
cation data table 610 holds and outputs identification 
data. Identification data embedding unit 609 embeds 
the identification data in the image data in frequency 
domain and outputs the resultant data. Inverse discrete 
cosine transforming unit 81 1 transforms the image data 
in frequency domain into data in space domain and out- 
puts the resultant image data. The image data in fre- 
quency domain may be converted into the image data in 
space domain by fast Fourier transforming method 
rather than the discrete cosine transforming method. 

Selecting unit 808 alternatively selects the output 
signal of buffer 807 or the output signal of inverse dis- 
crete cosine transforming unit 81 1 . When the output sig- 
nal of buffer 807 is selected, the original image data is 
output. When the output signal of inverse discrete 
cosine transforming unit 81 1 is selected, the image data 
in which the identif ication data is embedded is output. 

Since the embedding method of the identification 
data according to the fourth example is the same as that 
of the third example, the description thereof is omitted. 
To extract the identification data, the discrete cosine 
transforming means orthogonally transforms the image 
data in space domain into the image data in frequency 
domain. Thereafter, a subtracting unit (not shown) sub- 
tracts the frequency spectrum of the original image data 
from the frequency spectrum in which the identification 
data is embedded and obtains the frequency spectrum 
of the identification data. 

As explained above, according to the present inven- 
tion, since both image data with electronic watermark 
data and image data without electronic watermark data 
can be encoded by one encoding system rather than 
two encoding systems, the hardware scale can be 
remarkably reduced. 

According to the present invention, even if image 
data is illegally copied, it can be identified. This is 
because identification data has been embedded in the 
image data. Thus, by detecting the identification data, 
the route of the illegal copy can be tracked. 

In addition, when identification data is deleted or 
destroyed and thereby original image data thereof is ille- 
gally copied, the image quality of the image data 
remarkably deteriorates. Thus, the image data can be 
prevented from being illegally forged and copied. 

Although the present invention has been shown 



and explained with respect to a best mode embodiment 
thereof, it should be understood by those skilled in the 
art that the foregoing and various other changes, omis- 
sions, and additions in the form and detail thereof may 
5 be made therein without departing from the spirit and 
scope of the present invention. 

Claims 

10 1. An image data encoding system for embedding 
electronic watermark data to an original image, 
which comprises: 

discrete cosine transforming means (101) for 
75 discrete-cosine transforming said original 

image; 

electronic watermark data embedding means 
(102) for embedding said electronic watermark 
data (103) in the data which has been trans- 
20 formed by said discrete cosine transforming 

means (101); and 

data selecting means (106; 110) for selecting 
the output signal (107) of said discrete cosine 
transforming means (101) or the output signal 
25 (108) of said electronic watermark data 

embedding means (102). 

2. The image data encoding system as set forth in 
claim 1 , which further comprises: 

30 

a flip-flop (111) connected to said data select- 
ing means (106), 

wherein said data selecting means (106; 110) 
selects the output signal (107) of said discrete 
35 cosine transforming means (101) or the output 

signal (108) of said electronic watermark data 
embedding means (102) corresponding to 
information stored in said flip-flop (111). 

40 3. The image data encoding system as set forth in 
claim 1 or 2, 

wherein said data selecting means (106; 110) 
selects the output signal (107) of said discrete 
cosine transforming means (101) or the output sig- 
45 nal (1 08) of said electronic watermark data embed- 
ding means (102) corresponding to an external 
signal (112). 

4. An image data encoding system for encoding digital 
50 image data (401) in a predetermined encoding 
manner and outputting the resultant image data, 
which comprises: 

an electronic watermark embedding means 
55 (404) for embedding electronic watermark data 

selected from a plurality of types of electronic 

watermark data (408(0), 408(1), 408(2) 

408(n)) to said digital image data (401), 
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wherein at least one of the plurality of types of 
electronic watermark data (408(0),.. .,408(n)) is ! 
predetermined electronic watermark data 
which does not affect said digital image data 
(401) even if embedded in said digital image s 
data (401). 

An image data encoding system for encoding digital 
image data (401) in a predetermined manner and 
out-putting the resultant data, comprising: io 

a plurality of electronic watermark data tables 
(408(0), 408(1), 408(2),..., 408(n)) having a plu- 
rality of types of electronic watermark data for 
identifying said digital image data (401); is 
an electronic watermark data selecting means 
(407) for selecting one of said electronic water- 
mark data tables ; and 

an electronic watermark data embedding 
means (404) for embedding the selected type 20 
of electronic watermark data in said digital 
image data (401), 

wherein at least one of said electronic water- 
mark data tables (408(0),.. .408(n)) has a pre- 
determined electronic watermark data which 25 
does not affect said digital image data (401) 
even if embedded in the digital image data 
(401). 

The image data encoding system as set forth in 30 
claim 4 or 5, which further comprises: 

transforming means (402) for transforming said 
digital image data (401) into frequency compo- 
nents and outputting the resultant data to said 35 
electronic watermark data embedding means 

(404) ; 

a quantizing means (405) for quantizing the 
data in which electronic watermark data has 
been embedded by said electronic watermark 40 
data embedding means (404); and 
a variable-length encoding means (406) for 
encoding output data of said quantizing means 

(405) into variable-length code. 

45 

The image data encoding system as set forth in 
anyone of the foregoing claims, 
wherein said predetermined electronic watermark 
data is composed of other than random numbers 
generated by an algorithm corresponding to a nor- so 
mat distribution. 

An image inputting apparatus, which comprises: 

image pickup means (501; 601) for obtaining 55 
an analog image signal; 
analog-to-digital converting means (502; 602) 
for converting said analog image signal 



obtained by said image pickup means (501; 
601) into image data; 

transforming means (503; 603) for transforming 
said image data into data (701) in first fre- 
quency domain; 

storing means (507; 607) for temporarily stor- 
ing the data (704) in said first frequency 
domain; 

identification data holding means (510; 610) for 
holding identification data (704); 
means (509; 702) for adding said identification 
data (704) to the data (701) in the first fre- 
quency domain and generating data (703) in 
second frequency domain; and 
selecting means (508; 608) for selecting either 
of the data (704) in said first frequency domain 
and the data (703) in said second frequency 
domain and outputting the selected data. 

9. The image inputting apparatus as set forth in daim 
8, wherein said transforming means (503; 603) is 
an orthogonal transforming means. 

10. An image inputting apparatus, comprising: 

image pickup means (601) for obtaining an 
analog image signal; 

analog-to-digital converting means (602) for 
converting said analog image signal obtained 
by said image pickup means (601) into image 
data; 

transforming means (603) for transforming said 

image data into data (701) in first frequency 

domain; storing means (807) for temporarily 

storing said image data; 

identification data holding means (610) for 

holding identification data (704); 

means (702) for adding the identification data 

(704) to the data (701) in said first frequency 

domain and generating data (703) in second 

frequency domain; 

inverse-transforming means (811) for inversely 
transforming the data in said second frequency 
domain into data in time domain; and 
selecting means (808) for selecting either of 
the output signal of said inverse-transforming 
means (81 1) and the output signal of said stor- 
ing means (807). 

11. The image inputting apparatus as set forth in daim 
10, 

wherein said transforming means (503; 603) is an 
orthogonal transforming means and said inverse 
transforming means (81 1) is an orthogonal inverse- 
transforming means. 

12. The image inputting apparatus as set forth in any- 
one of the claims 8 to 1 1, which further comprises: 
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compressing means (504; 604) for compress- 
ing and encoding the output signal of said 
selecting means (508; 608; 808). 



10 



15 



20 



25 



30 



35 



40 



45 



SO 



55 



10 



EP 0 859 337 A2 



F I G.1 



1 



103 



ELECTRONIC 
WATERMARK DATA 



ORIGINAL IMAGE DATA 

^101 



DISCRETE COSINE 
TRANSFORMING MEANS 



107- 



102 



ELECTRONIC WATERMARK 
DATA EMBEDDING MEANS 



108- 



EL 



106 



DATA SELEC 


5TING MEANS 


•i 




QUANTIZING MEANS 




(105 


ENCODIN 


IG MEANS 



T 

MPEG DATA 



EP 0 859 337 A2 



F IG.2 



103 



1 



ELECTRONIC 
WATERMARK DATA 



ORIGINAL IMAGE DATA 



1 



101 



DISCRETE COSINE 
TRANSFORMING UNIT 



107-/"; 



102 



ELECTRONIC WATERMARK 
DATA EMBEDDING UNIT 



108 



110 



SELECTING UNIT 



1 



111 



FLIP-FLOP 



f 104 



| QUANTI2 


:iNG UNIT 




<105 


j ENCODING UNIT 



t 

MPEG DATA 



1? 



EP 0 859 337 A2 



t 



F I G. 3 



1 



103 



ELECTRONIC 
WATERMARK DATA 



ORIGINAL IMAGE DATA 

,101 



DISCRETE COSINE 
TRANSFORMING UNIT 



107" 



1 



102 



ELECTRONIC WATERMARK 
DATA EMBEDDING UNIT 



108- 



SELECTING UNIT 



1 



104 



QUANTIZING UNIT 



SELECT SIGNAL 



,105 



ENCODING UNIT 



T 



] 



MPEG DATA 



112- 



EP 0 859 337 A2 



F I G.4 



i 



407 



ELECTRONIC 
WATERMARK DATA 
SELECTING MEANS 



1 



408(0) 



ELECTRONIC 
WATERMARK 
DATA TABLE 



1 



408(1 ) *\ ^408(2) 



ELECTRONIC 
WATERMARK 
DATA TABLE 

T - 



408(n) 



401 



ORIGINAL IMAGE DATA 

402 



DISCRETE COSINE 
TRANSFORMING MEANS 



1 



404 



ELECTRONIC WATERMARK 
DATA EMBEDDING MEANS 





^405 


QUANTIZING MEANS 




^406 


ENCODIN 


IG MEANS 



MPEG DATA 
\ 



409 



410 
BUFFER 



14 



EP 0 859 337 A2 



F I G 



. 5 




IMAGE PICKUP MEANS 1 




r < 502 


ANALOG-TO-DIGITAL 
CONVERTING MEANS 




r <503 



TRANSFORMING MEANS 



^510 




/509 


V , 507 


IDENTIFICATION 

DATA 
HOLDING MEANS 




IDENTIFICATION 
DATA 
EMBEDDING MEANS 




STORING MEANS 









\ i <508 

DATA SELECTING I 
MEANS I 



{504 

QUANTIZING MEANS 

| (505 
ENCODING MEANS 



1? 



EP 0 859 337 A2 



F I G. 6 



601 



CCD IMAGE PICKUP 
DEVICE 



I 



i 



602 



ANALOG-TO-DIGITAL 
CONVERTING UNIT 



] 



1 



603 



DISCRETE COSINE 
TRANSFORMING UNIT 



610 



609 



IDENTIFICATION 
DATA TABLE 


— >- 


IDENTIFICATION DATA 
EMBEDDING UNIT 







1 607 



SELECTING UNIT 




I 



i 



604 



QUANTIZING UNIT 



I 



1 



605 



VARIABLE-LENGTH 
ENCODING UNIT 



16 



EP 0 859 337 A2 



o 



s s 

II 

^ CO 



ZD 

tr 
I- 

o 

LU 

Q_ 

CO 

>- 

o 
zz. 

LU 
ZD 

o 

LU 
CC 



< 

< 
Q 

Z 

o 

I— 

< 
o 



1- 
z: 

LU 
Q 



* 

CD 



CM 



O 



I— 

o 

LU 
Q_ 
CO 

>- 
O 

-z. 

LU 
ZD 

a 

LU 

cr 

LU 

o 
< 




< 
o 

z 

g 

< 
o 



LU 
Q 



g 
o 



EP 0 859 337 A2 



F IG. 8 



601 



CCD IMAGE PICKUP 
DEVICE 

\ ^602 

ANALOG-TO-DIGITAL 
CONVERTING UNIT 



i 



610 



IDENTIFICATION 
DATA TABLE 



1 



603 



DISCRETE COSINE 
TRANSFORMING UNIT! 



A 



609 



IDENTIFICATION DATA 
EMBEDDING UNIT 



1 



811 



INVERSE-DISCRETE 
COSINE CONVERTING UNIT 



1 



807 



BUFFER 



Li 



808 



SELECTING UNIT 



18 



EP 0 859 337 A2 



F I G. 9 



901 



IMAGE PICKUP MEANS 



,902 



ANALOG-TO-DIGITAL 
CONVERTING MEANS 



1 



903 



CONVERTING MEANS 



I 



,904 



QUANTIZING MEANS 



I 



905 



VARIABLE-LENGTH 
ENCODING MEANS 



I 



i 



906 



SCRAMBLING MEANS 



EP 0 859 337 A2 



F I G.10 



ORIGINAL IMAGE DATA 
,,1020 



.1010 



DISCRETE COSINE 
TRANSFORMING MEANS 



r 



ELECTRONIC 
WATERMARK DATA 



1033 



ELECTRONIC WATERMARK 
| DATA EMB EDD I NG MEA NS 



1030-' 




1032 



1 



1031 



CONSTANT OC 



1034 



J 



FREQUENCY COMPONENTS OF IMAGE DATA IN 
WHICH ELECTRONIC WATERMARK DATA IS INSERTED 



FIG. 11 

OBJECTIVE IMAGE DATA ORIGINAL IMAGE DATA 



i 



1120 



DISCRETE COSINE 
TRANSFORMING MEANS 



r 



L ELECTRONIC WATERMARK 
DATA EXTRACTING MEANS 



1 



1110 



DISCRETE COSINE 
TRANSFORMING MEANS 



ELECTRONIC 
WATERMARK DATA 

^1150 




1131 



1132 



1 



1 



1140 



INNER PRODUCT 
CALCULATING MEANS 



1 



1160 



STATISTICAL SIMILARITY 
DETERMINING MEANS 



20 



EP 0 859 337 A2 



i 



F I G. 12 



.1203 



ELECTRONIC 
WATERMARK DATA 



ORIGINAL IMAGE DATA 

/1201 



DISCRETE COSINE 
TRANSFORMING MEANS 



i 



1202 



ELECTRONIC WATERMARK 
DATA EMBEDDING MEANS 



1 



1204 



QUANTIZING MEANS 



I 



1 



1205 



ENCODING MEANS 



I 



MPEG DATA 



This Page Blank (uspto) 



i 



CO 

< 

co 

CO 

o> 
m 

CO 

o 

Q. 
LU 



(19) 



J) 



Europdisches Patentamt 
European Patent Office 
Office europeen des brevets 




(12) 



(88) Date of publication A3: 

27.01.1999 Bulletin 1999/04 

(43) Date of publication A2: 

19.08.1998 Bulletin 1998/34 

(21) Application number: 98102571.1 

(22) Date of filing: 13.02.1998 



(id EP 0 859 337 A3 

EUROPEAN PATENT APPLICATION 

(51) int. Cl. 6 : G06T 11/00, G06T 1/00 



(84) Designated Contracting States: 

AT BE CH DE DK ES Fl FR GB GR IE IT LI LU MC 
NL PT SE 

Designated Extension States: 
AL LT LV MK RO SI 

(30) Priority: 14.02.1997 J P 29992/97 
17.02.1997 JP 32212/97 
12.03.1997 JP 57469/97 

(71 ) Applicant: NEC CORPORATION 
Tokyo (JP) 



(72) Inventors: 

• Nakano, Hi rota ka 
Minato-ku, Tokyo (JP) 

• Hashimoto, Masahiro 
Minato-ku, Tokyo (JP) 

• Manabe, Koji 
Minato-ku, Tokyo (JP) 

(74) Representative: 

von Samson-Himmelstjerna, Friedrich R., Dipl. 
Phys. et al 

SAMSON & PARTNER 
Widenmayerstrasse 5 
80538 Munchen (DE) 



(54) Image data encoding system and image inputting apparatus 

(57) An image data encoding system is disclosed, 
that comprises a discrete cosine transforming means 

(101) for discrete-cosine transforming the original 
image, an electronic watermark data embedding means 

(102) for embedding the electronic watermark data 

(103) in the data that has been transformed by the dis- 
crete cosine transforming means (101), and a data 
selecting means (106; 1 10) for selecting the output sig- 
nal (107) of the discrete cosine transforming means 
(101) or the output signal (108) of the electronic water- 
mark data embedding means (102). 

Another image data encoding system is also dis- 
closed, that comprises an electronic watermark embed- 
ding means (404) for embedding electronic watermark 
data selected from a plurality of types (408(0) t ..,408(n)) 
of electronic watermark data to the digital image data 
(401), wherein at least one of the plurality of types 
(408(0),.. -,408(n)) of electronic watermark data is pre- 
determined electronic watermark data that does not 
affect the digital image data (401) even if the electronic 
watermark data is embedded in the digital image data 
(401). 
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